Papers with automatic methods
Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing low-cost approaches to build a high-quality functioning dialogue agent are limited to a few widely-spoken languages. |
| Approach: | They propose automatic methods that use ToD training data to build a functioning agent in another language . they compare the method to existing methods that only use a small training set . |
| Outcome: | The proposed method improves the state-of-the-art in Chinese to English transfer using zero-shot data compared to existing full-shot methods . the proposed method achieves 46.7% and 22.0% in task success rate and dialogue success rate, respectively. |
Author Commitment and Social Power: Automatic Belief Tagging to Infer the Social Context of Interactions (N18-1)
Copied to clipboard
| Challenge: | Social power is a difficult concept to define, but is often manifested in how we interact with one another. |
| Approach: | They employ extra-propositional semantics extraction within NLP to study author commitment . they find that subordinates use significantly more instances of non-commitment than superiors . |
| Outcome: | The proposed method shows that subordinates use significantly more instances of non-commitment than superiors, and that enriching lexical features with commitment labels captures important distinctions in social meanings. |
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection (2021.acl-short)
Copied to clipboard
| Challenge: | a lack of labeled, non-English resources for hate speech detection limits research on hate speech . a recent study shows that zero-shot, cross-lingual learning models cannot be used as they are . lack of consistency limits research, and lack of models for non-english languages limits learning . |
| Approach: | They propose a zero-shot, cross-lingual transfer learning framework for hate speech detection . they use benchmark data sets in English, Italian, and Spanish to detect hate speech . |
| Outcome: | The proposed framework can't be used as it is, but needs to be carefully designed, the authors say . they find that non-hateful, language-specific taboo interjections are misinterpreted as signals of hate speech . |
Adversarial Domain Adaptation for Duplicate Question Detection (D18-1)
Copied to clipboard
| Challenge: | Recent years have seen the rise of community question answering forums . duplicate questions easily become ubiquitous as users often ask the same question, possibly in a slightly different formulation, making it difficult to find the best (or one correct) answer. |
| Approach: | They propose to use domain adaptation to detect duplicate questions in forums . they find that domain adaptation improves performance over multiple pairs of domains . |
| Outcome: | The proposed approach improves 5.6% over the best baseline across multiple pairs of domains. |
COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic (2021.acl-long)
Copied to clipboard
| Challenge: | a new method for fact-checking is needed to detect disinformation on the web . a dataset COVID-Fact contains 4,086 claims concerning the COVId-19 pandemic . |
| Approach: | They propose a FEVER-like dataset COVID-Fact of 4,086 claims concerning the COVId-19 pandemic . they automatically detect true claims and their source articles and generate counter-claims using automatic methods . |
| Outcome: | The proposed method reduces the cost of building domain-specific datasets for detecting misinformation . the proposed dataset contains 4,086 claims concerning the COVID-19 pandemic . |
Evaluating the Robustness of Discrete Prompts (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods that generate discrete prompts from a small set of training instances have reported superior performance, but manual writing prompts that generalize well is challenging due to several reasons. |
| Approach: | They propose to use discrete prompts to learn lexical constructs that would not be encountered in manually-written prompts. |
| Outcome: | The proposed method is robust against perturbations to NLI inputs but sensitive to other types of perturbations such as shuffling and deletion of prompt tokens. |
Ensuring Readability and Data-fidelity using Head-modifier Templates in Deep Type Description Generation (P19-1)
Copied to clipboard
| Challenge: | Existing generative methods overlook grammatical structure or make factual mistakes in generated texts. |
| Approach: | They propose a template-based method to ensure the readability of generated type descriptions . they also propose measurable metrics to measure the readibility of the generated type description . |
| Outcome: | The proposed method improves substantially compared with baselines and achieves state-of-the-art performance on both datasets. |
Gold: A Global and Local-aware Denoising Framework for Commonsense Knowledge Graph Noise Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to construct CSKGs with large semantic coverage are expensive and introduce spurious noise. |
| Approach: | They propose a denoising framework that incorporates entity semantic information, global rules, and local structural information from the CSKG. |
| Outcome: | The proposed framework outperforms baseline methods in noise detection tasks on synthetic noisy CSKG benchmarks. |
Adapting Entities across Languages and Cultures (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a structured knowledge base adapts named entities using their shared properties. |
| Approach: | They propose automatic methods to adapt named entities using shared properties . they compare them to human adaptations using a new dataset of human adaptation data . |
| Outcome: | The proposed methods compare to human adaptations using a new dataset. |
How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on instruction-tuned Large Language Models has used automatic methods based on text overlap and LLM judgments as cost-effective alternatives to human evaluation. |
| Approach: | They perform a meta-evaluation of automatic methods and assess their reliability across a broad range of tasks. |
| Outcome: | The proposed method is unreliable in free-form generation tasks and cross-lingual scenarios. |
Augmenting word2vec with latent Dirichlet allocation within a clinical application (N19-1)
Copied to clipboard
| Challenge: | Existing models that combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer’s disease from transcripts of picture descriptions are not suitable for clinical binary text classification tasks. |
| Approach: | They propose three models that combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer’s disease from transcripts of picture descriptions. |
| Outcome: | The proposed models outperform word2vec and LDA models on a clinical binary text classification task. |
Offensive Language and Hate Speech Detection for Danish (2020.lrec-1)
Copied to clipboard
| Challenge: | a growing number of social media platforms are detecting and dealing with offensive language . a recent study found that the best performing system for English is best for Danish . |
| Approach: | They propose automatic methods to detect offensive language on social media platforms . they use user-generated comments from various social media sites to find offensive language . |
| Outcome: | The proposed system performs best for both English and Danish language . it achieves a macro averaged F1-score of 0.74 and a best for Danish achieves 0.73 . |
Contextualized Graph Representations for Generating Counter-Narratives against Hate Speech (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Hate speech (HS) is a widespread problem in society with severe repercussions at both personal and societal levels. |
| Approach: | They propose to incorporate conversational history into CNs to confront biases and stereotypes driving hateful narratives. |
| Outcome: | The proposed strategies outperform existing methods on comparing graphical and text representations with varying degrees of context. |
Assessing the State of the Art in Scene Segmentation (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in scene segmentation have made it difficult to detect scenes in literary texts. |
| Approach: | They propose to modify existing models to improve detection of scenes in literary texts . they propose to use a training sample generation scheme to alleviate this problem . |
| Outcome: | The proposed model is more robust to different types of texts, while its overall performance is slightly worse than that of BERT-based models. |
Data Augmentation for Low-Resource Keyphrase Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on keyphrase generation rely on large-scale annotated datasets, which are not easy to acquire. |
| Approach: | They propose to use full text to improve keyphrase generation in resource-constrained domains by using the full text of the articles to augment their methods. |
| Outcome: | The proposed methods improve both present and absent keyphrase generation on three datasets and show that they are cost-effective. |
Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisation (2023.acl-long)
Copied to clipboard
| Challenge: | Argument summarisation is a promising but currently under-explored field. |
| Approach: | They propose a framework to generate key points from short texts in a task known as Key Point Analysis. |
| Outcome: | The proposed framework improves state-of-the-art in argument summarisation with performance improvement of 14 percentage points compared to ROUGE and human evaluation scores. |
MMAD:Multi-modal Movie Audio Description (2024.lrec-main)
Copied to clipboard
| Challenge: | Current methods of creating accessible movies rely on manual work, resulting in high costs and limited scalability. |
| Approach: | They propose a multi-modal movie audio description pipeline that generates narrations of information that is not accessible through unimodal hearing in movies. |
| Outcome: | The proposed pipeline surpasses existing baselines in performance on widely used datasets. |
Is This a Bad Table? A Closer Look at the Evaluation of Table Generation from Text (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing measures for table quality evaluation fail to capture the overall semantics of tables . failure to accurately assess table quality can result in including subpar content or overlooking valuable tables in documents. |
| Approach: | They propose a method that captures table semantics by breaking down a table into atomic statements and comparing them with ground truth statements. |
| Outcome: | The proposed method shows stronger correlation with human judgments of table quality across four datasets. |